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Abstract 

The Friedman's urn model is a popular urn model which is widely used in many dis- 
ciplines. In particular, it is extensively used in treatment allocation schemes in clinical 
trials. In this paper, we prove that both the urn composition process and the allocation 
proportion process can be approximated by a multi-dimensional Gaussian process al- 
most surely for a multi-color generalized Friedman's urn model with non-homogeneous 
generating matrices. The Gaussian process is a solution of a stochastic differential equa- 
tion. This Gaussian approximation together with the properties of the Gaussian process 
is important for the understanding of the behavior of the urn process and is also useful 
for statistical inferences. As an application, we obtain the asymptotic properties includ- 
ing the asymptotic normality and the law of the iterated logarithm for a multi-color 
generalized Friedman's urn model as well as the randomized-play-the-winner rule as a 
special case. 
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1 Introduction. 



Urn models have long been recognized as valuable mathematical apparatus in many areas 
including physical sciences, biological sciences and engineering (Johnson and Kotz, 1977; 
Kotz and Balakrishnan, 1997). Urn models are also extensively applied in clinical studies. 
The applications are mostly found in the area of adaptive design which is utilized to provide 
a response-adaptive allocation scheme. In clinical trials, suppose patients accrue sequen- 
tially and assume the availability of several treatments. Adaptive designs are inclining to 
assign more patients to the better treatments, while seeking to maintain randomness as a 
basis for statistical inference. Thus the cumulative information of the response of treatments 
on previous patients will be used to adjust treatment assignment to coming patients. For 
this purpose, various urn models have been proposed and used extensively in adaptive de- 
signs (Wei and Durham (1978), Wei (1979), Flournoy and Rosenberger (1995), Rosenberger 
(1996), Bai and Hu (1999,2005)). One large family of randomized adaptive designs is based 
on the Generalized Friedman's Urn (GFU) model (also named as Generalized Polya Urn 
(GPU) in literature). For more detailed reference, the reader is referred to Flournoy and 
Rosenberger (1995), Rosenberger (1996), Hu and Rosenberger (2006). 

A general description of the GPU model is as follows. Consider an urn containing 
particles of d types, respectively representing d 'treatments' in a clinical trial. At the 
beginning, the urn contains Iq = (^oi; ■ ■ -i ^od) particles, where Yofc > denotes the number 
of particles of type k, k = I, . . . ,d. At the stage m, m = 1, 2, . . ., a particle is drawn form 
the urn and replaced. If the particle is of type k, then the treatment k is assigned to the 
mth patient, k = 1, . . . ,d. We then wait for observing a random variable ^(m), the response 
of the treatment at the patient m. Here ^(m) may be a random vector. After that, an 
additional D]^ q{m) particles of type g, g = 1, . . . , d, are added to the urn, where Dk q{'m) is a 
function of $,{m) and also may be a function of urn compositions, assignments and responses 
of previous stages. This procedure is repeated through out n stages. After n draws and 
generations, the urn composition is denoted by the row vector = (l^i, . . . ,Ynci), where 
Ynk stands for the number of particles of type k in the urn after the nth draw. This relation 
can be written as the following recursive formula: 
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where = (l?fe^g(m))^ and is the result of the nth draw, distributed according 
to the urn composition at the previous stage, i.e., if the mth draw is type k particle, then 
the A;th component of is 1 and other components are 0. The matrices D^s are named 
as the addition rules. Furthermore, write Nn = {Nni, ■ ■ ■ , Nnd), where Nnk is the number 
of times a type k particle drawn in the first n stages. In clinical trials, Nnk represents the 
number of patients assigned to the treatment k in the first n trials. Obviously, 

n 

Nn = ^Xn. (1.2) 

fc=l 

In clinical applications, Y^-i/ Ylk=i ^n-i,k the probabilities of the patient n being allo- 
cated to treatments, and Nn/n are sample allocation proportions. The asymptotic behavior 
of Yn and is of immense importance (Hu and Rosenberger, 2003, 2006). Obviously, the 
asymptotic behavior of Yn and JV„ will depend on the addition rules Dm, especially the 
conditional expectations i?^ = {E.[Dk,q{m)\j^rn-i])'l for given the history sigma field 
^m-i generated by the urn compositions Yi, . . . , Ym-i, the assignments Xi, . . . , Xm-i and 
the responses ^(1), . . . , ^{m — 1) of all previous stages, m= 1,2, The conditional expec- 
tations H^s are named as the generating matrices. In some usual cases, the addition 
rules are assumed to be independent of the previous process. Thus, we may define is 
the expectation of the rule matrix D^,- For more generality, in the sequel of this paper, 
we define if^ to be the conditional expectation of D^^ when the history sigma field ^m-i 
is given, also we assume that at the stage m, the adding rule Dm is independent of the 
assignment Xm when the history sigma field is given. 

When Dm, m = 1,2, ... , are independent and identical distributed, the GFU model is 
usually said to be homogeneous. In such case Hm = H are identical and nonrandom, and 
usually the addition rule D^, is merely function of the mth patient's observed outcome. In 
the general non-homogeneous cases, both and Hjn depend on the entire history of all 
previous trials which provides more information of the efficacy of the treatments. Interesting 
examples of non-homogeneous urn models and their applications can be found in Andersen, 
Faries and Tamura (1994) and Bai, Hu and Shen (2002). 

Athreya and Karlin (1967, 1968) first considered the asymptotic properties of the GFU 
model with homogeneous generating matrix and conjecture that N„ is asymptotically nor- 
mal. This conjecture has not been solved for almost three decades until Janson (2004) 
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and Bai and Hu (2005) solved it independently. Janson (2004) established functional limit 
theorems of Y„ and N„ for the homogenous case by using the theory of continuous-time 
branching processes. Bai and Hu (2005) established the consistency and the asymptotic 
normality of the non-homogeneous GFU model by applying the central limit of martingales 
and the matrix theory. However, the asymptotic variances of Y„ and N„ are complicated 
and not easy to be understood. 

In the two-arm clinical trial, Bai, Hu and Zhang (2002) showed that the urn process 
{y^} with nonhomogeneous generating matrices H^'s can be approximated by a Gaussian 
process almost surely under some suitable conditions, where Yn = Yni represents the number 
of type 1 balls in the urn after the rath draw. As an application, the weak invariance principle 
and the law of the iterated logarithm for {1^} are established. However, the results for the 
allocation proportion Nni/n is not obtained. In this paper, we consider the general multi- 
color case. The strong approximation of the process {Yn, iV„) are established. In particular, 
the asymptotic normality and the law of the iterated logarithm for the multi-dimensional 
process (1^, iV„) are obtained. We will prove that under some mild conditions, the process 
(Yfi, N„) can be approximated by a multi-dimensional Gaussian process which is a solution 
of a simple multi-dimensional stochastic differential equation. This differential equation 
and the behavior of the Gaussian process make us to understand the complex asymptotic 
variances and the asymptotic behavior of Y^ and iV„ more easily. 

The approximation theorems will be presented in Section 2 whose technical proofs are 
stated in the last section. Some important properties of the limit processes are given in Sec- 
tion 3. By combining the approximation theorems and the properties of the limit processes, 
important asymptotic properties including the asymptotic normality and the law of the it- 
erated logarithm of Y^, and iV„ are derived in Section 4. Throughout this paper, C, Cg, etc. 
denote positive constants whose values can differ in different places, logx = ln(eVa;). For a 
vector X, \\x\\ denote its Euclidean norm, and ||M"|| = sup{||a5M"|| : ||a;|| = 1} for a martix 
M. Also we denote a„ = 1^1' to be the total number of balls in the urn after stage ra. 
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2 Strong approximation. 

In this section, we will give our main results on the Gaussian approximation of both the 
urn composition y„ and the allocation numbers Nn- We first need two assumptions on 
the addition rules Dm- We let = o'iYi, . . . ,Ym,Xi, . . . . . . ,^{m)) be history 

sigma field, and Hm = ^[Dm\^m-i] be the generating matrix. 

Assumption 2.1 Suppose there is a t > such that the generating matrices H^. satisfy 

n 

J2 \\Hm - H\\ = o(n^/2-^) a.s., (2.1) 

m=l 

with H bing a deterministic matrix and 

d 

Hqk > for k ^ q and Hqk = s for allq = 1, . . . ,d, 

k=l 

where Hg/^ is the {q,k)-entry of the matrix H and s is a positive constant. Without loss of 
generality, we assume s = 1 through out this paper. For otherwise, we may consider Ym/ s, 
Hm/s instead. 

Assumption 2.2 Let 

Vqki{n)=: Cov[{Dqk{n),Dqi{n))\^n-i\: q,k,l = l,...d 
and denote by Vnq = {Vqki{n))f. i^^. Suppose for some < e < 1/2, 

E\\Dnf^^<C <oo for all n = l,2, (2.2) 

n 

Vmq = nVq + o{n^~^) o.s., for all q = 1, . . . , d, (2-3) 

m=l 

where Vq = {Vqki)f. i^i, q = 1, . . . ,d, are d x d non-negative definite matrices. 

By Assumption 12. 11 H has a maximal eigenvalue 1 and a corresponding right eigenvector 
1 = (1, . . . , 1). Let A2, . . . ,Xd be other d — 1 eigenvalues of H. Then H has the following 
Jordan form decomposition 

T-^HT = diag{l,J) and J = diag{J2, . . . , Js) (2.4) 
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with 



Jf = 



At 1 
Xt 




1 



1 



(2.5) 



... Xt 
... 

where T = {t'l, . . . , t'^) and ti = 1. Denote hy p = max{i?e(A2), . . . , Re{X2)}, where Re{Xk) 
is the real part of the complex number Afc. And denote the order of Jt by Pt and v = 
max{ft : Re{Xt) = p}. Let v be the left eigenvector of H associated with the positive 
maximal eigenvalue 1 and satisfy vV = 1. We denote H = H — I'v, 5^1 = diag{v) — v'v, 
S2 = Ylq=i and S = if'Eiif + S2- For a d-dimensional Brownian motion {Wj;* > 0} 
with a CO- variance A, we denote the solution of the equation: 



St = Wt + 



ds, t>0, So = 



Equl 



by {St = Solut(Equl, Wt);t > 0}. In the next section, we will show that St is well defined 
if p < 1/2, and 



where, for any t>0 and any matrix M, is defined to be 



H 



k=0 



Also we denote the solution of the equation: 
St = Wt-Wi+ J' 



SM 



ds, t>0, Si = 



Equ2 



hy {St = Solut(Equ2, Wt);t > 0}. And we will show that St is well defined and 



St = J^ W,^Q"ds + Wt-Wit'' = J^{dWs)Q 
Now let Bti and Bt2 be two independent d-dimensional standard Brownian motions. 



Define Gu = Solut(Equl, St^sV^) and Gu = Solut(Equ2, StiS^'^), i = 1,2. Let Gt = 
GtiH + Gt2 and Gt = GtiH + Gt2. Then Gt = Solut(Equl, SaSj/^// + Bt2^l^^) and 
Gt = Solut(Equ2, Bti-^Y^H + 5*2 ^^a^^). 

The next two theorems are on the strong approximation. 



1/2 ^ , _ 



Theorem 2.1 Suppose p < 1/2. Under Assumptions \2.1\ and \2.^ there are two independent 
d— dimensional standard Brownian motions Bti and Bt2 (possibly in an enlarged probability 
space with the process (1^, Nn) being redefined without changing the distribution^ such that 
for some 7 > 0, 

Yn-nv = GniH + Gn2 + o{n^'^-^''^)a.s., (2.6) 
Nn-nv = Gni + / ^ dx{I -l'v)+o{n^/^-^'^^) a.s., (2.7) 

where r A 7 = min{r, 7}. 

Theorem 2.2 Suppose p = 1/2 and Assumptions \2. 1\ and \2.S\ are satisfied. Further, assume 
that 

°° II / / H-ll 

m=l 

Then there are two independent d— dimensional standard Brownian motions Bti and Bt2 
(^possibly in an enlarged probability space with the process (1^, Nn) being redefined without 
changing the distribution ) such that 

Yn-nv = GniH + Gn2 + 0{n^''^\og''-^n)a.s. (2.9) 

Nn-nv = Gni+ dx(I -I'v) + 0(n^/^log''-^n) a.s. (2.10) 

Ji X 

(dti + dx{I - 1'^)) H = GtiH + Gt2 - St2S2^'. (2.11) 



Also, 



Remark 2.1 The condition h2. <^|) is used by Bai and Hu (2005) to obtain the asymptotic 
normality. It is easily seen that it implies the condition 12. 1\) with r = 0. Bai and Hu (2005) 
also assumed that i^ml' = 1' o-nd Hm H . 

Remark 2.2 By I12.9\) - [2A1\) . under the assumptions in Theorem \2.2[ 

Yn-nv = {Nn - nv)H + 0{n^''^ log^"^ n) a.s. if v > I. 

The proof of Theorems 12.11 and 12.21 will be given in the last section. Before that, we give 
some properties of the limit processes and several application of these approximations. 
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3 Properties of the limit processes. 

This section will give several properties of the solutions of equations (Equl) and (Equ2) 
and Gaussian processes Gt, St, etc. By combining these properties with the approximation 
theorems in the above section we can obtain important properties of the urn models which 
will be given in the next section. The properties of the Gaussian processes will be also used 
in the proofs of the approximation theorems. 
We first need two lemmas. 

Lemma 3.1 Under Assumption \2. 11 there exists a constant C such that for any a > 1, 



1^11 < CaPloR^-^a. 



(3.1) 



Proof It is obvious that H = Tdiag{0, J)T-^ := TJT'^. It follows that 



So it is enough to show that for any a > 1, Ha"^* || < Ca^'^^^^^ log''* ^ a. Denote Jt = Xfl + It 
where 



Then lla-^' 



{a^'a^'W < Ca^^^^-'^'Wa^^W. Obviously, 



and It 







^0 

0. It follows that 



1 ... 
... 



'^O 1 

1 

. . . 

\^0 

\ 



... o\ 

... 

1 

... 0/ 



1 


















/,(lna) 



fe=o 



k\ 



k=0 



kl 



. 1 

. 

. 

. 

. 



(3.2) 



Then 

\\a^ < CJ2 ^^<C {log ar-'. 
k=0 

Hence ()3.ip is proved. 

Lemma 3.2 For any a >0, the equation 

Zt= [ —Hds, Za = 

. n S 



or equivalently 



dZt = ^Hdt, Za = (3.3) 



has an unique solution Zt = 0. 

Proof It is obvious that H = H — I'v has the Jordan form decomposition 

T^HT = diag{0, J2, . . . , J^) 

and ()3.3p is equivalent to 

ZtT = ^diag{0,J2,...,Js)dt, Z^T = 0. 
On the other hand, for each s, 

) = ^ z(^) = 

has an unique solution Z\ = 0. The proof is completed. 

From this Lemma, it follows that the solutions of (Equl) and (Equ2) are unique. The 
following two propositions tells us that the solutions exist. 

Proposition 3.1 Let {Wi\t > 0} he a d- dimensional Brownian motion with a co-variance 
matrix A. Suppose that Assumption \2. 1\ is satisfied and p < 1/2. Then the unique solution 
St = Solut{Equl,Wt) of the equation (Equl) is 

St= f W,— {-)''dx + Wt. (3.4) 

JO XX 

Also 

rt 

-H\^H 



St= [J^idW,)x-")t^ a.s. (3.5) 
Furthermore, with probability one St is continuous on [0, 00). 



Proof Fist, since ||x ■'^H < Cx ''(logx -"^ for < a: < 1 by Lemma \2>.1\ and W; 



a.s. 



X 



0(Y^xloglog3; ^) as x — > 0, we have 

W^x-" = 0{l)x^/^-P{logx-^Y U.S., 
W^ — (-)^ = 0{l)x-^^^-''(logx-^Y a.s. 

X X 

as X ^ 0. Since —1/2 — p > —1 and Wx^{^)^ is continuous on (0, cxd), it follows that the 
integral Wx^{-^)^ dx exists, and then St in (13. 4p is well defined and 

fwx-[-fdx=( fwxd{x-^)Y 

Jo X X \ Jq / 

= -Wxx-^\it" + ( 1^ diWx)x-^y^ =-wt+[ 1^ d{Wx)x-^y^. 

It follows that (13. 5p is true. Now we show that St is the solution of equation (Equl). Note 
that 

St = 0(1) f\xiogiogx-^)^/\t/xy{iog{t/x)y~Ux = o{i)t^/^-p{iogt-^y 

Jo 

as t —>■ oo. It follows that Sq = 0, the integral Jq ^ds exists and 
s Jo s Jo x ^x' Jq s 



t — ft , — ft w 

WxX-"-^dx / Hs^-^ds+ / —ds 

Jx Jo S 

'Wxx~^'^dxs^\l+ f^ds 
^ Jo ^ 

' Wxx'^-^dxt^ - f^dx+ f^dx= fwx-i-)"dx. 

Jo X Jo S Jq X^x' 



Then 

C^ds= fwx-{-fdx = St-Wt. 

Jo S Jq X 'X' 

So, St is the solution of equation (Equl). Finally, the continuity of St follows from the 
continuity of the Brownian motion Wf 

Proposition 3.2 Let {Wt]t > 0} be a d- dimensional Brownian motion with some co- 
variance matrix. Suppose Assumption \2.1\ is satisfied. Then the unique solution St = 
Solut(Equ2,Wt) of the equation (Equ2) is 

St= f Wx — i-Y'dx + Wt-Wit". (3.6) 

Ji X ^x' 
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Also 



St={ 1^ {dW^)x-" )t" a.s. 



(3.7) 



Furthermore, with probability one St is continuous on (0, oo) . 
Proof The proof is similar to Proposition 13.11 and so omitted. 

Proposition 3.3 Let {Wt;t > 0} 6e a d- dimensional Brownian motion with some co- 
variance matrix. Suppose Assumption \2. 1\ is satisfied. If p < 1/2, then for St = Solut(Equl,Wt) 
we have 

(3.8) 



/•n Q n—1 Q 

/ 2ldt=y ^+0{1) a.s. 
Jn t m 
•"J m=l 



If p < 1, then for St = Solut{Equ2, Wt) we have 

n-l 



/ ^dt = y ^+o{i) 

J-i t ^ m 

•^^ m=l 



a.s. 



(3.9) 



Proof We only give a proof of (j3.9p since the proof of (|3.8p is similar. First, from ()3.7p it 
follows that for all t > 1, 



|Var(5t) 



*(-)^'Var(Wi)(-)^dx 

I X X 



< C / {t/xfPlog'^''-\t/x)dx 



< < 



Ctlog^^~U, ifp = l/2 
Ct, if p < 1/2 

Ct'^P log"^"-^ log t, ifp>l/2 

So, E\\St\\ < CtP'^hog"''^/'^ t. According to equation (Equ2), 



> < Ct2(pvi) iog2--i t. 



St-Ss = Wt-Ws + 



— — dx, t > s > 1. 



It follows that 



(^-^]dt 

, ., ,„ V t m 

m=l 



' m 



£ / s,(i_l)dt+£ / 

oo ^ 

+ E- 



■dt 



m 



- ^"^+1 Wt - Wm 

m 



dt 



m=l 



S^H 

X 



m J m 



dxdt. 
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The first and the third term above are a.s. convergent because 

f; r'E||5,i||l-lMt<cf:""^^'"*-f''^'"'^"<c. 



m=l in=l 

and 



Z — J ml I T ^ — ^ 



m=l -^"^ 2: 

The second term is a.s. convergent because it is an infinite series of independent normal 
random variables with 

< 00. 



m=l ^■^^ ^ m=l 



It follows that 



m=l 



The proof of (|3.9|) is completed. 

Propositions 13. II and [3T2] give the solutions of equations (Equl) and (Equ2). To give 
further properties of the Gt and Gt-, we need the analytic representation of the solutions. 
Recall T = (t'^, . . . , t^), where ti = 1. Let {W^; t > 0} be a d-dimensional Brownian motion 
with some co-variance matrix A. First we consider (Equl). Let {St = Solut(Equl, Wj); t > 
0} be the solution of (Equl) and Ut = StT. Then Ut is the unique solution of the equation 

Ut = WtT + / t > 0, J7o = (3.10) 

Jq s 

Note that J = diag{0,J) = diag{0, J2, ■ ■ ■ , Js), where Jj's are defined as in (j2.5p . Write 



Ut = {Uti,Uf'^\ . . . ,uj:'^^), where U^^^ = {U^i , . . . , U^ll'.) is the vector which contains 
coordinate variables corresponding to Jj. Also write T = {l',T^'^\ . . . , T^*)), where T^*) = 
{t''^, . . . ,t'-^.) is the X d matrix which contains Vi columns of T corresponding to J,,. 
Obviously, U^j^ = Ut,i+u2+...+Ui.i+j and Uj = ti+u2+...+u^-i+j- It is easily seen that (13.10p is 
equivalent to 

Un = Wtl' 

duil^ = d{Wtt[,) + X,^dt, c/«=o, 

du^;^ = d{Wt^^) + ^ + \^dt, t/« = o, 

j = 2, i = 2,...,s. 
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On can show that the solution of equation (j3.1ip is 

Uti = Wtl' 

j = 2, . . . i = 2, - • • ,s. 

Putting all the f/'s to St = UtT~^, we obtain the solution of (Equl). 
Similarly, we have St = UtT^^, where 



Ut = WtT - WiT + 



'U.J 



1 s 



ds t > 0, Ui=0 



and, Ut = {Ut,M\- ■ ■ , Uh, Uf^ = iuil\ ■ ■ ■ , U^^), 
Uti= {Wt-Wi)l', 

j = 2, . . . i = 2, - • • ,s. 

Proposition 3.4 Under Assumption \2.1\ and p < 1/2, 

Var{ {GtiH + Gt2, Gti + f ^dx{I - I'v)) ] = tV 

Jo X 



with 



r= Var{{GuH + Gi2,Gii+ C ^dx{I - I'v))] 

Jo X 



r(2i) r(22) 



and 



r(22) 



1 /IX i^' 



'i3"'l] 1 ii" + S2 



dx, 



1 ^^^H' 



Si ( - ) dx 

X 



+ ii-i'vy /'[Tim'' J sj/ 

Jo Jx y ^xJ 



r(i2) ^p(i2) ^ 



1 ^^^H' 



dy 

H 

Si ( ^ 1 dx 



y \xJ 



dx{I 



+ H') 



1 /IX 



H 



Ssl^) dx{I-l'v). 
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Proof Let {W^; t > 0} be a d-dimensional Brownian motion with some co- variance matrix 
A, {St = Solut(Equl, Wt);t > 0} be the solution of (Equl). Notice {T-'^^'^WTt,t > 0} and 
{Wt,t > 0} are identical distributed. So {T-'^/^STt,t > 0} and {St,t > 0} are identical 
distributed. Hence (j3.15p is true. By 



Jo ^ Jo 



1 fy 
y Jo 



H 



dy= / dW^ 





X y ^xJ 



It follows that 



Var{5i} 







Coy {St, Ss] 



Var<; ^dy 



V^; 



X ^' Ax ^dx. 



dx = s{ -] VarlSi}, t > s, 



. y \xJ 



y \xJ 



dx. 



CoyiSi,j^ ^dy 



^ Cov{5i,5J 



dy 



1 ,^^H' 



y Jo \y 



dyVar{Si} 



--{I - H')-'yar{Si}. 



The proof is now completed by noticing the independence of Gti and Gt2- 



Proposition 3.5 Under Assumption \2. 1\ and p = 1/2, the limit 

f = lim t-\logt)^-^''Var{{GtiH + Gt2,Gti + 
exists, and 



'^dx{I-l'v))} 



p(ll) p(12) 
f(21) f(22) 



where 



(T*r(")T)ij 

(T*f(22)T)ij- 



+ |A;|-2tiiS2t' 



~ ((i^~l)!)^ 2u~ 



(3.17) 



(3.18) 



(3.19) 



whenever i = j = I + V2 + ■ ■ ■ + n and Re{Xi) = 1/2, ui = v, and {T*T^^'"'^T)ij = for 
otherwise u,v = 1, 2. Here a is the conjugate vector of a complex vector a. 
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Proof Let Wt be a d-dimensional Brownian motion with a co-variance A, St a solution of 
(Equ2) and Ut = StT. Then by UiSJSh and Proposition [321 

Uti = {Wt - Wi)l' =^ o(ti/2 log--i/2 1) 



and 



X X' 



L2 



.(i^/2log-V2,) + / W-.r»-(-)'Mog^'^Tfj.da; 

k=o 

1) + 1a. W.t(^^-{-)'^ log^ dx, 



where li is defined as in (j3.2p . It is easily seen that 



X X X 



So 



(i) L2 Aj 



0(1) rxi/2i(l)«KA,)^^gfc i 

Jl X X X 

0(tV2)^ if iie(A,) < 1/2, 

0(iV2iogfc+it), if i2e(A,) = 1/2. 



|^^T«i (-)^^ ( log^-i -)Tr"' dx + o(ti/2 \og--y^ t). 

X X X 



It follows that 



Al/i ^^4Kj)''log^-' J rf^, if ^e(A.) = 1/2, J 



Ui = 



otherwise. 



+ o(tl/2log--l/2i). 



Similarly, 



xj 1 L2 

ax = 



1 X 



1 r* 



/i log''"' J if = 1/2, J = 

0, otherwise. 



+o(ti/2log''-i/2t) 
On the other hand, if Re{\i) = Re{Xj) = 1/2 and I'i = vj = then 

Covj /V.4i(-)'Mog''-i-dx, /V.t;.ii(-)"Mog--i-dx| 

^Jl X X X Ji X X X J 

{2v - l)-i|Ar't^At -1(1 + o(l))tlog2^-^ t if Ai = A,-, 

[ (i + i)Trih-^^*^i(l + -(1))*^'"'-'^ log""' if ^ 
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It follows that 



lim t-\logt)^-^''yar\{GtiH + Gt2)T, (Gti + [ ^dxil - 1'v))t} 

t~*oo I Jl X J 



(11) 

\^a(2i) 

exists. Also \i i = j = 1 + V2 + ■ ■ ■ + vi and Re{\i) = 1/2, vi = then 



(3.20) 



(H(2i)) 



and (H(™))jj = for other cases, u,v = 1,2. The proof is completed. 
Proposition 3.6 Suppose Assumption \2. 1\ is satisfied. If p < 1/2, then for i = 1, 2, 



Gti = 0{tloglogty/^) and 



'G. 



■dx = 0(tloglogt)^/2) a.s. t —>■ oo. 



(3.21) 



(3.22) 



If p= 1/2, then fori = 1,2, 

Gti = 0((tlogloglogt)i/2iog«^-i/2^^ ^ ^ ^ 
/q = 0((tlogloglogt)i/2log''-i/2i) a.s. t oo. 

Proof Let Wt be a d-dimensional Brownian motion, St a solution of (Equl). If /> < 1/2, 
then by (j3.4p and Lemma |3. 11 

=o((tiogiogt)i/2)+ r 

JO 

=0((tloglogt)^/2) a.s. 



Ojxlog logxy/'^ .ty^^ 

■ W 



which implies ()3.2ip . 

When p = 1/2, let St be a solution of (Equl). Then Ut = StT a solution of (l3l^ . It 
is easily seen that (cf. Bai, Hu and Zhang 2002) 



0((tloglogt)i/2) a.s., if i?e(Ai) < 1/2 

0((tlogloglogt)i/2log^/2^) a.s., if i?e(Ai) = 1/2 



as t ^ oo. From (f3l^ it follows that, if Re{Xi) < 1/2, 

= 0((tloglogt)i/2) a.s. 
f/«= 0((tloglogt)V2) + t^eix.) jt 0(Wog^y/^) ^ o((tloglogt)V2) a.s. 

j — ■ ■ ■ 1 Vi] i = 1 , . . . , s , 
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and if Re{Xi) = 1/2, 

U^l^ = 0((tlogloglogt)i/2logi/2^) a.s. 

= 0((tlogloglogt)V2logV2^)^^l/2J^f 0((xlogloglogx)V2iog.-i-i/2,) 
= 0((tlogloglogt)l/2iogj-l/2^^ 

j = 2,...,Vi;i = l,...,s. 
It follows that St = UtT-^ = 0((t logloglogt)^/^ ^1^^^ 

/■* /"* 0((xl0gl0gl0gx)l/2l0g^-l/2 2;) 

/ — dx = / dx 

Jl X Ji X 

So IK22h is proved. 

4 Applications. 

In this section, we give several applications of the approximation theorems. First, by comb- 
ing Theorem 12.11 with Proposition 13.41 and Theorem 12.21 with Proposition 13.51 respectively, 
we have the following asymptotic normalities for (Yn,Nn)- 

Theorem 4.1 Under Assumptions \2.1\ and \2.2\ and p < 1/2, 

n-^/\Yn -nv,Nn- nv) ^ N{0,T), 

where T is defined in Proposition \3.4\ 

Theorem 4.2 Suppose Assumptions \2.1\ and \2.2\ are satisfied. Further, assume 12. 8\) is 
satisfied and p = 1/2. Then 

n^^/^ {log ny/^-" {Yn - nv, Nn - nv) ^ N{0,T), 

where T is defined in Proposition \3.5[ 



Also, by combining Proposition 13.61 with Theorem 12. II and Theorem 12.21 respectivelv. we 
have we have the following laws of the iterated logarithm. 
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Theorem 4.3 Suppose Assumptions \2. I\ and \2.^ are satisfied. If p < 1/2, then 

1^ — nt) = 0((nloglogn)"^/^) a.s., 

Nn — nv = 0[{nloglogn)^^'^) a.s. 
If 112. 8\) is satisfied and p = 1/2, then 

Yn — nv = O ((n log log log n)^/^ log'^^^/^ n) a.s., 

Nn — nv = 0(y{n log log log n)'^^'^ Xog^^^^"^ n) a.s. 



Next, we consider a two-treatment case in which the addition rule matrices are denoted 



by 



dliCml), 1 — dl('?ml) 
»^l-rf2(Cm2), d2(Cm2) 



where (^n, ^12), • • • , (Cni, '^n2) are assumed to be i.i.d. random variables with < (ifc(^mfc) < 
1 for k = 1,2. This is a generalized randomized play-the-winner (RPW) rule (Bai and Hu, 
1999). When S,mi and ^m2 are dichotomous and dk{x) = x, then generalized RPW model is 
the well-known RPW model proposed by Wei and Durham (1978). In using the generalized 
RPW rule, at the stage m, if the patient m is allocated to treatment 1 and the response 
^mi is observed, then di{^mi) balls of type 1 and 1 — di{^mi) balls of type 2 are added to 
the urn. And, if the patient m is allocated to treatment 2 and the response ^m2 is observed, 
then d2{Cm2) balls of type 2 and 1 — d2{Cm2) balls of type 1 are added to the urn. It is 
obvious that the generating matrix is 

/ \ 

Pi, qi 



\Q2, P2j 

where p^ = E.[dk{£,mk)] and Qk = ^ — Pk for A; = 1,2. It is easily checked that Assumptions 
[2?T] and [2r2] are satisfied, and vi = ^2/(^1 + 52), V2 = qi/{qi + ^2), Ai = 1, p = A2 = pi - q2- 
Denote aj = viV2 = j^f^ and = ^igf^, where au = Var(4(ei)) for k = 1, 2. Then 



af(l,-l)'(l,-l) 



an 



fl, 



a2(l,-l)'(l,-l) 



H = p{v2,-viyil,-l). 
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Further, it is trivial that, if Wt is a Brownian motion with a variance-covariance matrix 
(T^(l, — 1)'(1, — 1), then Wt = cr{Bt, —Bf) where Bt is a standard Brownian motion. When 
p < 1/2, multiplying 1' in both side of the equation (Equl) yields Stl' = 0, which implies 
St = {St, —St) and St is a solution of 

/"* S 

St = aBt + p —dx, So = 0. 



X 



It is easily check that 



and 



Also, 



St = cjtP [ x-PdB,^ = aBt + apt^ [ B^x-P~^dx 
Jo Jo 

[ -^dx = atP I Btx-P-^dx. 
Jo X Jo 

yar{St) = aH'^P f x'^Pdx = -^^t, 
^ ' Jo 1 - 2p ' 

[ fo ^^^} = fo fo^^ ^ y)^-^-'y-^-'^^dy = (T^^^^' 



Coy{St,Ss} = a^{-yy^r{Ss) = r—:r[-Ys,t> s, 

I Jo ] Jo (l-2p)(l-p) 

Hence by applying Theorem 12.11 we conclude the following theorem. 

Theorem 4.4 For the generalized RPW rule, if p = pi — q2 < 1/2, then there are two 
independent standard Brownian motion Bti and Bt2 such that for some 7 > 0, 

nn 

Yni - nvi =nP I x~Pd[paiBxi + 0-2 -63:2) + o(n^/^"'^) a.s., 
Jo 

Nni - nvi =ainP / x^PdB^i + cr2nP / Bx2X^''^^dx + o(n^/^"'^) a.s. 
Jo Jo 



and 



rn 

p{Nni - nvi) = nP x~Pd{paiBxi + 0-2-6^2) - o-2-Bn2 + o( 
Jo 



a.s. 



In particular, 

n 



n qi+q2 n 91 + 92 
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where 5] = {aij)fj^^ and 



0-11 



^22 



{pi - Q2fqiq2 + (gi + q2){aiq2 + a2gi) 

(l-2(pi-g2))(gi+g2)2 

91^2 + 2(aig2 + a2gi) 



0"12 = 0-21 



'{l-2{pi-q2)){qi+q2?' 

{pi - q2)qiq2 + (0192 + 02^1) 



{l-2{pi-q2)){qi+q2Y 

When p = pi — q2 = 1/2, by considering the equation (Equ2) and applying Theorem[27 
instead, we can define two independent standard Brownian motion Bti and Bt2 such that 

x-^/^dB^i+(T2n^'^ j B^2x'^'^dx + 0{^/n} a.s. 



Nni - nvi 



and 



-(A^'ni - nvi 



If we denote 



/n 1 
x-^/^d{-aiB,i + a2B,2) - {Bn2 - n^^^Bu) + 0{V^) a.s. 



1 



4 4(gi+g2)2 91 + ^2 



H : = 9192 + 2(aig2 + 0291] 



and 



B{t) = 1 [ JaiS.i + a2B,2) , 

c Ji ^ 

it is easily to check that B{t) is a standard Brownian motion. Hence we obtain the following 
theorem for the case of p = 1/2. 

Theorem 4.5 For the generalized RPW rule, if p = Pi — q2 = 1/2, then there a standard 
Brownian motion B{t) such that 



Yni - n- 



92 



Nni - n 



91 + 92 

92 

91 + 92 



=c7n^/^S(logn) + 0(v/n) a.s., 



--2an^/'^B{\ogn) + < 



0{^/n) in probability, 

0(n log log n)^/^ a.s. 



where = qiq2 + 2{qiq2 + 0291). In particular, 



1- >"ni - ng2/(9i + 92) 

limsup — = a a.s.. 



Y^2n(log n)(log log log 



n) 
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and 



y Nni-nq2/{qi+q2) „~ 

iimsup — y^^^^^^=^^^^^^= = la a.s. 

n^oo y'2n(logn)(logloglogn) 



,1/2 f^nl q2 Nnl q2 \ 



n^/^ 1^ A iV 0, E 

n qi+q2 n qi + q2 



where E = {aij)fj^^ and an = a^ , a^ = a2i = 2a^ , a22 = 4ct^- 

5 Proof of the approximation theorems. 

Define 

Mn2 = Tl=i Xm{Dm " E[£>^|^„_i]) =: ^2^^^ AM, 



(5.1) 



m2- 



Recall that a„ = 1^1' is the total number of balls in the urn after stage n. By (II. 1|) we have 

n 



k=l 



m=l 



+ {Xm - E[X™|^„„i] + ^^^)H + Xm{Hm " H)} 



n—1 n 



+ Mn2 + MniH + -^H + ^ - H) 



m=U m=l 

ra— 1 n 



m=0 m=l 

( sincei;„l' = a^, H = H - I'v, vH = 0) 
nv + Yo + Mn2 + MniH +J2{^-v)H 

m=l 

-1^ n-1 _ n 

+(^ - + V ^^^^(^ - ^)// + V - /f) 

n-l . , 

Kr,. - mv 



— H + Rni + YQ, 5.2 

m 

m=l 

where 

■y n-l _ ^ n 

i?ni = - + V -v)H+y2 X^{H^ - H). (5.3) 
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Also by (fT^ . 

n n n—l 

Nn = ^(X„-E[X^|#-^_i]) + J] E[X^|^^_i] =M„i+ ^ 

m=l m=l m=0 

n-1 

= nv + Mni+Y.{^-v){l-l'v) 

m=0 "™ 
m=l 

V — V 

m=l 
n-1 



+ M„i + V {I - I'v) + R^,, (5.4) 



r7i=l 



where 



Rn2 = {--v) + y2 ^^^^ (— - ^) - 1'^) • (5-5) 



«0 "1 flm 



The expansions given in (j5.2p and (|5.4p are the key component in asymptotic analysis 
of Yn and A?"„. Actually, if we neglect the remainder Rn\ and replace MniH + Mn2 by a 
Brownian motion Wn, then 



n-1 

— mv - 



L — ^ m 



m 

ni=l 



which is very similar to the equations (Equl) or (Equ2). We will show (y„ — ni?, Nn—nv) can 
be approximated by a 2d-dimensional Gaussian process by approximating the martingale 
{Mni, Mn2) to a 2(i-dimensional Brownian motion. First show that the remainders Rni 
and Rn2 can be neglected. 

Proposition 5.1 Under Assumptions h2.1\) and \2.S^) . we have for any 5 > {), 

n 

Rni =o{n^) + Xm{Hm - H) = o(nV2-) a.S., 

m=l 

Rni =o{n^) a.s. 



To proving this proposition, we need two lemmas, the first one can be found in Hu and 
Zhang (2004). 

Lemma 5.1 (Rn and Zhang (2004) ) If AQ,, = APn + Qn-iH/{n -l),n> 2, then 
WQnW = 0{\\Pn\\) + V ^^^^^(n/m)^log--i(n/m). 

m=l 
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Lemma 5.2 Suppose sup^ CUD^lp < oo. Under Assumptions \2. il 

Mn = 0(ni/2) and M„i =^ ©(n^/^), 
M„ =■ 0(n^/2+5) > Q M„i =• 0((n log log n)i/2)^ 

o„ - n =■ 0(ni/2+<5) > 0. 
Furthermore, under Assumption \2.2\. 

Mn '=' 0((nloglogn)^/^) and an — n 0((n log log n)^/^) . 

Proof Note that ||AM„i|| < ||X„|| + E[||X„|||^„_i] < 2, ||AM"„|| < + E[||£)„|||^„_i] 

and a„ = n + IqI' + -M„l' + Ylm=i ^m{Hm — H)l' . By the properties of martingale, the 
results follow easily. 

Lemma 5.3 Suppose p < 1/2 and sup^ E||Z)m|P < oo. Under Assumptions \2. 1\ 

— — V = o{n~^^'^~^^) a.s. for any 6 > 0. (5-6) 

On 

Proof By (j5.3p and Lemma 15.21 it is obvious that 

n— 1 I 1 n 

\\R^^\\ <CJ2 " + Yl 11^- - ^11 = o(n'/'+^) a.s. for any 6 > 0. 

m=l m=l 

From (j5.2p and Lemma 15.21 it follows that 

n-l 



m 

m=l 



By Lemma |5. 11 it follows that 

" „/'™l/2+5^ 

y„ - n« = o(ni/2+5) + y ^n/myiog''-\n/m) = oin^/^+^) a.s. 



m 

m=l 



Hence 



Yn Yn-nv , {Yn-nv)l' Yn ^y^^g 

V = 1 = of ) = o(n ^/^^") a.s. 

On n n an n 

(|5.6p is proved. 
Now, we tend to 

Proof of Proposition [5TT1 Notice - i?) = o(n"^+^^) a.s. by Lemma lOl 

The proof is completed by noticing (j5.3p and (|5.5p . 

The next result is about the conditional variance-covariance matrix of the 2d-dimensional 
martingale (iVf„i,M"„2). 
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Proposition 5.2 We have 



t[(AM„i)'AM^2|^m-i] = (5.7) 
and under Assumptions l^. 1\ and\KM 

n 

E[iAMmiyAMmi\^m-i] = nSi + o{n^-') a.s. i = 1,2. (5.8) 

m=l 

Proof (15. 71) is trivial. For (15. 8D. we have 



E[AM;2AM„2|^n-l] = ^[{Dn - Hn)'diag{Xn)[Dn - HnWn^l] 

= E[{Dn - Hn)'diag{^){Dn - Hn)\^n-l] 

a-n-1 

= E[{Dn- Hn)'diag{v){Dn- Hn)\-^n-l] 

+E[(D„ - Hn)'{diag{^) - diag{v)){Dn - Hn)\^n-i] 

d d Y 

Under Assumptions 12. H 12.21 by Lemma 15.31 we have 

n 



m=l 



Also 



E[AM;,iAM„i|^„_i] 

= EiX'M'^n-i] - (E[X„|^„_i])'e[X„|^„_i] 

Y' Y I Y Y' Y I 
= E[diag{Xn)\--^n-i] ^-^^ = diag{-^) 

= diag{v) - v'v + o(n"-^/^+'^) a.s. 

(I5.8|l is proved. 

Proof of Theorem 12.11 Suppose that Assumption 12.21 is satisfied. According to ()5.8p . 

n 

Y E[(AM^i,AM„2)'(AM„i,AM„2)|^m-i] = n dmff(Si, S2) + o(n^-^) a.s. 

m=l 

It follows from Theorem 1.3 of Zhang (2004) that, there exist two standard d-dimensional 
Brownian motions Bti and Bt2 for which 

(M:„i, M„2) - (-B„iS;/', Bn2T.y^) = o(ni/2-T) a.s. (5.9) 
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Here 7 > depends only on d and e. Without loss of generality, we assume 7 < e/3. 
Now let Gti = Solut{Equl, BuT.y^) {i = 1,2). Then by Proposition [331 



Jo X m 



^ 1 

m=l 

Write Gt = Gt2^^'^ + GaSy^iJ. Combining the above equality with ([521), and 
Proposition 15.11 yields 

Yn-nV- Gn = y ^rn-mV-Gm Jj ^ o(^l/2-rA7) 

^-^ m 

m=l 

By Proposition I5.H 

n j 1/2— tA7\ 

Yn-nv-Gn = o(ni/2-rA7) ^ '- {n / mY \og"-^ {n / m) = o{n^'^-^''^) a.s. 

^-^ m 

m=l 

Finally, combining the above equality with (j5.4p . (j5.9p and Proposition 15.11 vields 
N^-nv = M„i + ^'^^^i— ^(/-l't;) + o(n'^) 

m=0 

= S„iS}/V V^(/-l'^) + o(ni/2-A7) 

m=l 

Jo X 
Jo X 

The proof is now completed. 

Proof of Theorem [231 (HSj) remains true. Let Gu = Solut(Equ2, BuT.y'^) {i = 1, 2). 
Then by Proposition 13.31 

/ il£id^= y ^ + o(l)a.s. f = l,2. 
J, X m 

Write Gt = Gt2^^'^ + Gti^^'^H. Combining the above equality with ([521), dS^j) and 
Proposition 15.11 yields 

Y^-nv- Gn = V ^- jf + o(nV2-7) + V - //) a.s. 

771=1 m=l 
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By Proposition 15.11 

n I l/2-7^ 

Yn-nv -Gn = o(ni/2-7) + '-(n/m)^/'^ log^-^fn/m) 

m=l 

m=l m=l 

=o(ni/2) + 0(1) log'^-i n = 0[n^/^ \og^-^ n) a.s. 

j=i 

Finally, combining the above equality with (j5.4p . (j5.9|) and Proposition 15.11 yields 

^-^ m 

m=0 

n-1 ^ 
,1/2 , \ ^ '^r, 



m=l 

Gni + J^^^ dx{I - 1'-") + 0{n^/^ log^^i n) a.s. 



The proof is now completed. 
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